The HTML5 Canvas element is a powerful tool for displaying resolution-dependent bitmap data. It's great for efficiently rendering 2D graphics, images, artwork, or game components. My current interest in the Canvas element is to use it as the foundation for an interactive image viewer for my future projects within the IPython Notebook widget framework.
I knew early on that I would need to learn a lot about JavaScript to make this project work. JavaScript is a really curious programming language, a wilderness of bizarre constructs and third-party modules. Navigating through that mess is risky without a good reference book: JavaScript: The Good Parts. I was really surprised by much I had to learn about both RequireJS and BackboneJS before I could start making meaningful progress on the technical parts of my project. Ultimately the most satisfying moment for me was when the Module Pattern finally started making sense.
My initial goals for the Canvas Widget are to makes it as easy and intuitive to use as possible. The widget must accept as input an image from a Numpy array or from a traditional URL. It must support images as greyscale, RGB, or RGBA. The user must have the ability to define back-end Python callback functions to be triggered in response to front-end mouse events on the canvas.
This widget should be as easy (or as hard??) to install and use as the built-in widgets.
The Canvas Element is a great tool for displaying images and drawing artwork onto a bitmap surface. It has built-in support for size and rotation transforms, and good number of third-party JavaScript libraries have sprung up adding all kinds of amazing features.
The people at Mozilla have a nice description of the Canvas.
Comprehensive Canvas reference at whatwg.org specifically for developers: The Canvas Element.
Use Canvas to display an image.
The Canvas Element height and width properties were initially quite confusing for me. That's because its a two-part problem! 1) the Canvas has an inherent width and height measured in data pixels; 2) the Canvas as displayed to the screen is controlled by the element's CSS width and height. The StackOverflow discussion canvas-width-and-height-in-html5 was extremely helpful.
More on Canvas width and height: I found I could not rely on the style width and height to always default to the Canvas' inherent width and height properties after several changes. Maybe I did something wrong? Maybe something to do with the IPython Notebook environment? I don't know. My solution was to explicitly set the Canvas style width and height whenever I needed to modify the Canvas' inherent width and height.
Here's a great description of the order of events when drawing an image to the canvas.
Developing and installing widgets is made much easier now by using jupyter-pip extension to setuptools. Also check out ipython-widgetboilerplate for a comprehensive example on setting up a new widget project.
The built-in IPython parent widget class DOMWidget has methods for handling CSS styles. Use those methods instead of building custom handlers for particular style properties, e.g. border thickness and color.
The notebook about IPython's Traitlet Events gives a lot of insight into the relationship between front-end widgets and back-end data structures. IPython's built-in widgets generally connect a simple data type (an integer or float, a list of values, etc.) to a simple widget (slider, checkbox, dropdown list). This explains why most built-in widgets focus on synchronizing a value property between Python and JavaScript.
The built-in button widget is a departure from the above pattern. It doesn't manage any data, instead it's internal framework is adapted to manage custom click events generated by the frontend.
The built-in IPython DOMWidget has methods for handling CSS styles. Use those methods instead of building cstom handlers for particular style properties, e.g. border thickness and color.
In my application I don't have a clean correspondence between a Python back-end variable and a simple HTML5/JavaScript front-end widget, which is how the IPython built-in widgets are designed. Instead I have an image and related cropping/scaling and offset parameters. I don't want to perform extra work and update all widget parameters when a single model parameter is changed. Some update are low overhead, e.g. redraw the image, while others require a lot of work, e.g. receive new PNG-compressed image data and copy it into the internal Image object.
The best solution I've found to date is to forgo performing any work in my Backbone View's this.update() method, and instead use separate event handlers for each major part of my model. These can be defined in this.initialize() and it might look like this:
this.model.on('change:_src', this.update_src, this);
this.model.on('change:_width', this.update_width, this);
this.model.on('change:_height', this.update_height, this);
The method this.update() in my JavaScript derived class is called automatically by the BackboneJS framework for all change events triggered by IPython's Traitlet synchronization process. This is implemented by the parent class DOMWidgetView via the method initialize() with a statement similar to the following:
this.model.on('change', this.update, this);
In JavaScript I found I needed to call this.update() at the end of my this.render() method, even if my own this.update() doesn't do anything (or maybe doesn't even exist!). If I left out this call odd things would happen, e.g. a second view of my model would not reflect any prior CSS style properties applied to earlier view instances.
Here's how to check if a given variable exists inside the Backbone Model:
if (this.model.has('_src')) {
this.update_src();
}
StackOverflow discussion rendering/appending views
List of BackboneJS built-in events
Nice example handling mouse click events using BackboneJS.
Understanding Canvas Element mouse coordinates and events in the context of BackboneJS event handling.
Many more BackboneJS examples
In [ ]: